I recently started an AI-focused educational newsletter, that already has over 70,000 subscribers. TheSequence is a no-BS (meaning no hype, no news etc) ML-oriented newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. An image is worth a thousand words says the old wisdom quote that captures the importance of visual analysis in the learning process of human species. Every time we are presented with a visual scene, our brains make thousands of inferences about the objects in it and their contextual nature.

We propose a formulation of visual localization that does not require construction of explicit maps in the form of point clouds or voxels. The goal is to learn an implicit representation of the environment at a higher, more abstract level, for instance that of objects. To study this approach we consider procedurally generated Minecraft worlds, for which we can generate visually rich images along with camera pose coordinates. We first show that Generative Query Networks (GQNs) enhanced with a novel attention mechanism can capture the visual structure of 3D scenes in Minecraft, as evidenced by their samples. We then apply the models to the localization problem, investigating both generative and discriminative approaches, and compare the different ways in which they each capture task uncertainty. Our results show that models with implicit mapping are able to capture the underlying 3D structure of visually complex scenes, and use this to accurately localize new observations, paving the way towards future applications in sequential localization. Supplementary video available at https://youtu.be/iHEXX5wXbCI.

An image is worth a thousand words says the old wisdom quote that captures the importance of visual analysis in the learning process of human species. Every time we are presented with a visual scene, our brains make thousands of inferences about the objects in it and their contextual nature. For instance, if we see a person sitting down, we will infer that there is a chair underneath him. Visual inferences works even when we can't see the object. For instance, if we see a closet in a bedroom, we would assume there are clothing items inside even when we can't see them.